{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": true,
    "editable": true
   },
   "source": [
    "# Tensors\n",
    "## This script introduces various ways to create tensors in TensorFlow"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": true,
    "editable": true
   },
   "source": [
    "First we start with loading TensorFlow and reseting the computational graph."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {
    "collapsed": true,
    "deletable": true,
    "editable": true
   },
   "outputs": [],
   "source": [
    "import tensorflow as tf\n",
    "from tensorflow.python.framework import ops\n",
    "ops.reset_default_graph()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": true,
    "editable": true
   },
   "source": [
    "### Start a graph session\n",
    "Get graph handle with the tf.Session()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {
    "collapsed": false,
    "deletable": true,
    "editable": true
   },
   "outputs": [],
   "source": [
    "sess = tf.Session()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": true,
    "deletable": true,
    "editable": true
   },
   "source": [
    "### Creating Tensors\n",
    "TensorFlow has built in function to create tensors for use in variables.  For example, we can create a zero filled tensor of predefined shape using the `tf.zeros()` function as follows."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {
    "collapsed": true,
    "deletable": true,
    "editable": true
   },
   "outputs": [],
   "source": [
    "my_tensor = tf.zeros([1,20])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": true,
    "editable": true
   },
   "source": [
    "We can evaluate tensors with calling a `run()` method on our session."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {
    "collapsed": false,
    "deletable": true,
    "editable": true
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([[ 0.,  0.,  0.,  0.,  0.,  0.,  0.,  0.,  0.,  0.,  0.,  0.,  0.,\n",
       "         0.,  0.,  0.,  0.,  0.,  0.,  0.]], dtype=float32)"
      ]
     },
     "execution_count": 4,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "sess.run(my_tensor)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": true,
    "editable": true
   },
   "source": [
    "TensorFlow algorithms need to know which objects are variables and which are constants. The difference between these two objects will be explained later in the chapter. For now we create a variable using the TensorFlow function `tf.Variable()` as follows."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {
    "collapsed": true,
    "deletable": true,
    "editable": true
   },
   "outputs": [],
   "source": [
    "my_var = tf.Variable(tf.zeros([1,20]))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": true,
    "editable": true
   },
   "source": [
    "Note that you can not run `sess.run(my_var)`, this would result in an error.  Because TensorFlow operates with computational graphs, we have to create a variable intialization operation in order to evaluate variables.  We will see more of this later on.  For this script, we can initialize one variable at a time by calling the variable method `my_var.initializer`."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {
    "collapsed": false,
    "deletable": true,
    "editable": true
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([[ 0.,  0.,  0.,  0.,  0.,  0.,  0.,  0.,  0.,  0.,  0.,  0.,  0.,\n",
       "         0.,  0.,  0.,  0.,  0.,  0.,  0.]], dtype=float32)"
      ]
     },
     "execution_count": 6,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "sess.run(my_var.initializer)\n",
    "sess.run(my_var)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": true,
    "editable": true
   },
   "source": [
    "Let's first start by creating variables of specific shape by declaring our row and column size."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {
    "collapsed": true,
    "deletable": true,
    "editable": true
   },
   "outputs": [],
   "source": [
    "row_dim = 2\n",
    "col_dim = 3"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": true,
    "editable": true
   },
   "source": [
    "Here are variables initialized to contain all zeros or ones."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {
    "collapsed": true,
    "deletable": true,
    "editable": true
   },
   "outputs": [],
   "source": [
    "zero_var = tf.Variable(tf.zeros([row_dim, col_dim]))\n",
    "ones_var = tf.Variable(tf.ones([row_dim, col_dim]))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": true,
    "editable": true
   },
   "source": [
    "Again, we can call the initializer method on our variables and run them to evaluate thier contents."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {
    "collapsed": false,
    "deletable": true,
    "editable": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[[ 0.  0.  0.]\n",
      " [ 0.  0.  0.]]\n",
      "[[ 1.  1.  1.]\n",
      " [ 1.  1.  1.]]\n"
     ]
    }
   ],
   "source": [
    "sess.run(zero_var.initializer)\n",
    "sess.run(ones_var.initializer)\n",
    "print(sess.run(zero_var))\n",
    "print(sess.run(ones_var))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": true,
    "editable": true
   },
   "source": [
    "### Creating Tensors Based on Other Tensor's Shape\n",
    "If the shape of a tensor depends on the shape of another tensor, then we can use the TensorFlow built-in functions `ones_like()` or `zeros_like()`."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {
    "collapsed": false,
    "deletable": true,
    "editable": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[[ 1.  1.  1.]\n",
      " [ 1.  1.  1.]]\n",
      "[[ 0.  0.  0.]\n",
      " [ 0.  0.  0.]]\n"
     ]
    }
   ],
   "source": [
    "zero_similar = tf.Variable(tf.zeros_like(zero_var))\n",
    "ones_similar = tf.Variable(tf.ones_like(ones_var))\n",
    "\n",
    "sess.run(ones_similar.initializer)\n",
    "sess.run(zero_similar.initializer)\n",
    "print(sess.run(ones_similar))\n",
    "print(sess.run(zero_similar))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": true,
    "editable": true
   },
   "source": [
    "### Filling a Tensor with a Constant\n",
    "Here is how we fill a tensor with a constant."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {
    "collapsed": false,
    "deletable": true,
    "editable": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[[-1 -1 -1]\n",
      " [-1 -1 -1]]\n"
     ]
    }
   ],
   "source": [
    "fill_var = tf.Variable(tf.fill([row_dim, col_dim], -1))\n",
    "sess.run(fill_var.initializer)\n",
    "print(sess.run(fill_var))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": true,
    "editable": true
   },
   "source": [
    "We can also create a variable from an array or list of constants."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {
    "collapsed": false,
    "deletable": true,
    "editable": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[8 6 7 5 3 0 9]\n",
      "[[-1 -1 -1]\n",
      " [-1 -1 -1]]\n"
     ]
    }
   ],
   "source": [
    "# Create a variable from a constant\n",
    "const_var = tf.Variable(tf.constant([8, 6, 7, 5, 3, 0, 9]))\n",
    "# This can also be used to fill an array:\n",
    "const_fill_var = tf.Variable(tf.constant(-1, shape=[row_dim, col_dim]))\n",
    "\n",
    "sess.run(const_var.initializer)\n",
    "sess.run(const_fill_var.initializer)\n",
    "\n",
    "print(sess.run(const_var))\n",
    "print(sess.run(const_fill_var))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": true,
    "editable": true
   },
   "source": [
    "### Creating Tensors Based on Sequences and Ranges\n",
    "We can also create tensors from sequence generation functions in TensorFlow.  The TensorFlow function `linspace()` and `range()` operate very similar to the python/numpy equivalents."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {
    "collapsed": false,
    "deletable": true,
    "editable": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[ 0.   0.5  1. ]\n",
      "[ 6  9 12]\n"
     ]
    }
   ],
   "source": [
    "# Linspace in TensorFlow\n",
    "linear_var = tf.Variable(tf.linspace(start=0.0, stop=1.0, num=3)) # Generates [0.0, 0.5, 1.0] includes the end\n",
    "\n",
    "# Range in TensorFlow\n",
    "sequence_var = tf.Variable(tf.range(start=6, limit=15, delta=3)) # Generates [6, 9, 12] doesn't include the end\n",
    "\n",
    "sess.run(linear_var.initializer)\n",
    "sess.run(sequence_var.initializer)\n",
    "\n",
    "print(sess.run(linear_var))\n",
    "print(sess.run(sequence_var))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": true,
    "editable": true
   },
   "source": [
    "### Random Number Tensors\n",
    "We can also initialize tensors that come from random numbers like the following."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {
    "collapsed": false,
    "deletable": true,
    "editable": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[[ 1.1772728   1.36544371 -0.89566803]\n",
      " [-0.02099477 -0.17081328  0.2029814 ]]\n",
      "[[ 2.54200077  1.42822504  1.34831095]\n",
      " [ 2.28473616  0.36273813  0.70220995]]\n"
     ]
    }
   ],
   "source": [
    "rnorm_var = tf.random_normal([row_dim, col_dim], mean=0.0, stddev=1.0)\n",
    "runif_var = tf.random_uniform([row_dim, col_dim], minval=0, maxval=4)\n",
    "\n",
    "print(sess.run(rnorm_var))\n",
    "print(sess.run(runif_var))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": true,
    "editable": true
   },
   "source": [
    "### Visualizing the Variable Creation in TensorBoard\n",
    "To visualize the creation of variables in Tensorboard (covered in more detail in Chapter 11), we will reset the computational graph and create a global initializing operation."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {
    "collapsed": false,
    "deletable": true,
    "editable": true
   },
   "outputs": [],
   "source": [
    "# Reset graph\n",
    "ops.reset_default_graph()\n",
    "\n",
    "# Start a graph session\n",
    "sess = tf.Session()\n",
    "\n",
    "# Create variable\n",
    "my_var = tf.Variable(tf.zeros([1,20]))\n",
    "\n",
    "# Add summaries to tensorboard\n",
    "merged = tf.summary.merge_all()\n",
    "\n",
    "# Initialize graph writer:\n",
    "writer = tf.summary.FileWriter(\"/tmp/variable_logs\", graph=sess.graph)\n",
    "\n",
    "# Initialize operation\n",
    "initialize_op = tf.global_variables_initializer()\n",
    "\n",
    "# Run initialization of variable\n",
    "sess.run(initialize_op)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": true,
    "editable": true
   },
   "source": [
    "We now run the following command in our command prompt:\n",
    "\n",
    "`tensorboard --logdir=/tmp`\n",
    "\n",
    "And it will tell us the URL we can navigate our browser to to see Tensorboard. The default should be:\n",
    "\n",
    "`http://0.0.0.0:6006/`"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": true,
    "editable": true
   },
   "source": [
    "![Variable_in_Tensorboard](https://github.com/nfmcclure/tensorflow_cookbook/raw/master/01_Introduction/images/02_variable.png)"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.5.2"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}
